This invention relates to an image processing apparatus, an image processing method, and an image processing system for executing color conversion for input color image data so that the color image data falls within the color reproduction range of an output device.
Image processing includes color conversion processing of executing color conversion for color image data so that the color image data falls within the color reproduction range of the device for visualizing the color image data. For example, to output a color image to a CRT or a printer, not all colors can be represented and only colors within the color reproduction range of the device are output. When input color image data has a portion beyond the color reproduction range, color conversion processing is required for converting unreproduced colors beyond the color reproduction range into reproducible colors.
Hitherto, various methods have been designed for color conversion as to how colors not reproduced in an image output device are reproduced, and some software products that can execute color conversion have a function for such a purpose. Representative techniques of such color conversion processing include a conversion method for holding gradation, that for holding hue, and that for holding lightness.
The conversion method for holding gradation is a method of changing not only colors outside the color reproduction range, but also colors in the color range of color image data at one rate for compression so as to place all colors within the color reproduction range. This method holds color differences and therefore holds color change such as gradation. Thus, it is an optimum method as color conversion for an image such as a photo attaching importance to gradation. However, when colors within the color reproduction range in which an original image can be reproduced faithfully are changed largely, the method may be inferior in color reproducibility as a result.
In the conversion method for holding hue or lightness, only colors outside the color reproduction range are changed to boundary colors of the color reproduction range with hue or lightness held, whereby whole color change is eliminated and only unreproducible colors are converted. However, if colors resulting from changing the colors outside the color reproduction range or colors close to the resulting colors already exist in the original image, discrimination between the colors and the original colors is degraded. For example, if a part of a gradation portion is outside the color reproduction range, the colors in the part of gradation become the same color by the conversion and the gradation is lost. For another example, the original image portions different in color become the same color and the colors cannot be distinguished from each other.
Thus, the conversion methods have merits and demerits and a conversion method for producing good results for every image is not available. Therefore, in the current mainstream techniques, generally a method of adopting the conversion method considered to be the most favorable for the original type is often used. For example, if the original is an image such as a photo, importance is attached to gradation and the conversion method for holding gradation is adopted; if the original is a graph used in business, etc., importance is attached to lightness and the conversion method for holding lightness is used.
However, if the conversion method is thus selected according only to the original type, the demerits of the selected conversion method are reflected on the output image intact, thus the user may feel dissatisfaction. For example, if the original is determined to be an image attaching importance to gradation, such as a photo, and color conversion is executed by the conversion method for holding gradation, reproducible colors are also converted into different colors, thus a problem remains in color reproducibility. If the conversion method for holding hue or lightness is selected for executing conversion, different colors of the original image become the same, as a result of the conversion and cannot be distinguished from each other. For example, there is a possibility that a problem may occur such that although a map is colored, the colors cannot be distinguished from each other as a result of conversion or that although a gradation portion should exist in a part, flat color representation results. These problems are caused by the fact that the color conversion method is selected according only to the original type; there may be a problem in that the image contents are not considered.
For example, a system described in the Unexamined Japanese Patent Application Publication No. Hei 5-167839 counts the number of pixels outside the color reproduction range.
If the ratio of the number of pixels outside the color reproduction range is small, the conversion method for holding hue is adopted; if the ratio of the number of pixels within the color reproduction range is small, the conversion method for holding gradation is adopted. Further, a system described in the Unexamined Japanese Patent Application Publication No. Hei 6-162181 divides a color space into blocks for each hue and counts the number of pixels outside the color reproduction range for each block. If a block with a large ratio of the count exists, the conversion method for holding gradation is used; otherwise, the conversion method for holding hue is used. In the systems, the conversion method is selected for each block. Thus, even if the original is a photo, a part of the gradation portion becomes flat or even if the original is an image with large flat portions, the original colors are not reproduced if a large number of pixels are beyond the color reproduction range.
Described in the Unexamined Japanese Patent Application Publication No. Hei 5-227418 are a method of counting the number of pixels outside the color reproduction range, preparing a histogram, and using the conversion method for holding gradation so as to place 90% pixels of the count in the color reproduction range, a method of determining conversion scaling interactively, and the like. Problems similar to those described above also exist in the art described in the Unexamined Japanese Patent Application Publication No. Hei 5-227418.
For example, a color image converter described in the Unexamined Japanese Patent Application Publication No. Hei 7-203234 divides an equal color space into unit areas, finds the number of pixels contained in each unit area, maps unit areas outside the color reproduction range to other color unit areas constant in lightness in response to the number of pixels, and executes color conversion. According to the art, conversion of the colors outside the color reproduction range to the same colors as other used colors is lessened, thus gradation color change is represented in different colors and if the original image is represented in different colors, the image resulting from conversion are also represented in different colors; degradation of color discrimination can be reduced. Further, since the colors within the color reproduction range are represented in intact colors in most cases, unnecessary color change is prevented and color reproducibility is enhanced. However, since color mapping is executed only for the unit areas outside the color reproduction range, for example, for continuous gradation from the colors within the color reproduction range to the colors outside the range, color continuity is impaired due to color change in portions outside the color reproduction range and desired gradation cannot be represented.
Thus, in the prior arts, it may be impossible to represent gradation portions as gradation and represent flat portion colors in appropriate colors for the flat portions; it is difficult to provide an output image satisfactory to the user.
On the other hand, when the user criticizes the color difference between input and output images, for example, he or she of ten takes out a part of the image and comments on the color difference and discrimination degradation. The image portions whose color difference is discussed are often flat tone portions and comparison such that some colors in the gradation portions differ is not much made. In contrast, the difference between colors placed in the neighborhood becomes a comparison object in gradation portions in many cases. For example, if a gradation portion becomes almost flat tone as discrimination is degraded, it becomes a factor unsatisfactory to the user.
Thus, different criteria are applied to flat tone and gradation portions even in one image and if the same color conversion is executed, either must be sacrificed. Optimum color conversion may be executed in response to parts of an image in such a manner that different conversion methods are applied to the flat tone and gradation portions. However, prior art for such purpose does not exist as things stand now.
It is therefore an object of the invention to provide an image processing apparatus, an image processing method, and an image processing system for performing color processing in response to an input image so as to represent a gradation portion as gradation and perform processing appropriate for flat color for a flat color portion at the same time.
According to the invention as in aspect 1, there is provided an image processing apparatus comprising means for inputting image data, color distribution recognition means for recognizing a color distribution in a color space and position information in an image space of the image data input through the input means, and means for determining one or more color reproduction parameters of the image data based on the color distribution and position information recognized by the color distribution recognition means and a color reproduction range of an output device.
According to the invention as in aspect 2, there is provided an image processing apparatus comprising means for inputting image data, color distribution recognition means for recognizing a color distribution of the image data input through the input means, grouping means for sorting the color distribution recognized by the color distribution recognition means into one or more groups, and means for determining a color reproduction parameter of the image data based on the groups into which the color distribution is sorted by the grouping means and a color reproduction range of an output device.
According to the invention as in aspect 3, there is provided an image processing apparatus comprising means for inputting image data, color distribution recognition means for recognizing a color distribution in a color space and position information in an image space of the image data input through the input means, grouping means for sorting the color distribution into one or more groups based on the position information recognized by the color distribution recognition means, and means for determining one or more color reproduction parameters of the image data based on the groups into which the color distribution is sorted by the grouping means and a color reproduction range of an output device.
In the invention as in aspect 4, in the image processing apparatus as in aspect 2 or 3, the color distribution recognition means divides the color space into areas and counts the number of pixels belonging to each division area and the grouping means determines whether or not adjacent areas can be combined based on the counts of the adjacent areas provided by the color distribution recognition means and groups the areas in response to the determination result.
In the invention as in aspect 5, in the image processing apparatus as in aspect 2 or 3, the color distribution recognition means divides the color space into areas, counts the number of pixels belonging to each division area, and recognizes pixel distribution information and the grouping means determines whether or not adjacent areas can be combined based on the counts of the adjacent areas provided by the color distribution recognition means and the pixel distribution information and groups the areas in response to the determination result.
In the invention as in aspect 6, in the image processing apparatus as in aspect 3, the color distribution recognition means divides the color space into areas, counts the number of pixels belonging to each division area, and relates position information of pixels belonging to each area and the grouping means determines whether or not adjacent areas can be combined based on the counts of the adjacent areas provided by the color distribution recognition means, determines positions of the pixels belonging to the adjacent areas, and groups the areas on the color space in response to the two determination results.
In the invention as in aspect 7, in the image processing apparatus as in aspect 2 or 3, the grouping means groups the areas except for color distributions of portions where no degradation of color discrimination is assumed to occur.
According to the invention as in aspect 8, there is provided an image processing apparatus comprising means for inputting image data, cluster preparation means for extracting areas on an image plane made up of similar color pixels based on the image data input through the input means and preparing clusters based on the extracted areas, cluster analysis means for recognizing color distributions of the clusters prepared by the cluster preparation means and preparing a group, and means for determining a color reproduction parameter of the image data for each of the clusters prepared by the cluster preparation means based on the group generated by the cluster analysis means and a color reproduction range of an output device.
In the invention as in aspect 9, the image processing apparatus as in aspect 8 further includes cluster correction means for again dividing and again integrating the cluster prepared by the cluster preparation means based on the group generated by the cluster analysis means to prepare new clusters, recognizing color distributions of the new clusters, and generating a new group, wherein the color reproduction parameter determination means determines a color reproduction parameter of the image data for each of the new clusters based on the new clusters and the new group generated by the cluster correction means and the color reproduction range of the output device.
In the invention as in aspect 10, in the image processing apparatus as in aspect 8, the cluster preparation means extracts clusters on the image space by clustering based on one or more feature amounts extracted from the image data input through the input means.
In the invention as in aspect 11, in the image processing apparatus as in aspect 8, the cluster preparation means divides the image data input through the input means into blocks on the image space, repeats block division and integration in response to dispersion of pixel values in the blocks for extracting a global similar color area, and extracts a detailed cluster by clustering based on the global similar color area.
In the invention as in aspect 12, in the image processing apparatus as in aspect 8, the cluster preparation means divides the image data input through the input means into blocks on the image space, repeats block division and integration in response to dispersion of pixel values in the blocks for extracting a global similar color area, checks the global similar color area for color distribution, determines whether or not cluster preparation using position information is required, and extracts a detailed cluster by clustering based on the global similar color area, if necessary, based on the determination result.
In the invention as in aspect 13, in the image processing apparatus as in aspect 8, the cluster analysis means recognizes a color distribution based on dispersion of pixels belonging to the cluster prepared by the cluster preparation means.
In the invention as in aspect 14, in the image processing apparatus as in aspect 8, the cluster analysis means recognizes a color distribution based on a color value change rate of adjacent pixels belonging to the cluster prepared by the cluster preparation means and dispersion of pixels belonging to the cluster.
In the invention as in aspect 15, in the image processing apparatus as in aspect 8, the cluster analysis means groups the clusters except for color distributions of portions where no degradation of color discrimination is assumed to occur.
In the invention as in aspect 16, in the image processing apparatus as in aspect 9, the cluster correction means integrates the clusters based on representative colors of the clusters and the color distributions in the clusters recognized by the cluster analysis section.
In the invention as in aspect 17, in the image processing apparatus as in aspect 9, the cluster correction means divides and integrates clusters in response to a form of a multi-dimensional histogram based on the color distributions and color value in the clusters recognized by the cluster analysis section.
In the invention as in aspect 18, in the image processing apparatus as in aspect 2, 3, or 8, the color reproduction parameter determination means comprises gradation determination means for determining spread magnitude of color change of the group for determining the color reproduction parameter based on the determination result of the gradation determination means.
In the invention as in aspect 19, in the image processing apparatus as in aspect 2, 3, or 8, the color reproduction parameter determination means determines a color compression method and color compression strength for the group based on a distribution form of a portion where a color portion space on the color space contained in the group is outside the color reproduction range.
In the invention as in aspect 20, in the image processing apparatus as in aspect 2, 3, or 8, the color reproduction parameter determination means determines a color compression method and color compression strength for the group based on a distribution form of a portion where a color portion space on the color space contained in the group is outside the color reproduction range and a distribution form of the group.
In the invention as in aspect 21, in the image processing apparatus as in aspect 19 or 20, the color space is an equal color space and the distribution form is determined based on any one of spread in a lightness direction, spread in a hue direction, or spread in a chroma direction or combination thereof in the color portion space on the equal color space contained in the group.
In the invention as in aspect 22, in the image processing apparatus as in aspect 1, 3, or 8, the color reproduction parameter determination means prepares one or more conversion parameters of a first equal color space to a second equal color space and one or more pieces of position mask information each indicating a position on the image space to which the conversion parameter is applied.
In the invention as in aspect 23, in the image processing apparatus as in aspect 2, 3, or 8, the color reproduction parameter determination means generates a group into which the groups are collected and determines the color reproduction parameter for the group.
In the invention as in aspect 24, in the image processing apparatus as in aspect 3 or 8, the color reproduction parameter determination means generates a group into which the groups are collected and determines a conversion parameter of a first equal color space to a second equal color space and position mask information indicating a position on the image space to which the conversion parameter is applied for the group.
According to the invention as in aspect 25, there is provided an image processing apparatus comprising means for inputting image data, one or more multi-dimensional color conversion tables for executing color conversion of the image data input through the input means based on preset lattice point data, color distribution recognition means for recognizing a color distribution in a color space and position information in an image space of the image data input through the input means, and means for determining the lattice point data for the one or more multi-dimensional color conversion tables based on the color distribution, the position is information, and a color reproduction range of an output device.
According to the invention as in aspect 26, there is provided an image processing apparatus comprising means for inputting image data, a multi-dimensional color conversion table for executing color conversion of the image data input through the input means based on preset lattice point data, color distribution recognition means for recognizing a color distribution of the image data input through the input means, grouping means for sorting the color distribution recognized by the color distribution recognition means into one or more groups, and means for determining the lattice point data for the multi-dimensional color conversion table based on the groups into which the color distribution is sorted by the grouping means and the color reproduction range of the output device.
According to the invention as in aspect 27, there is provided an image processing apparatus comprising means for inputting image data, one or more multi-dimensional color conversion tables for executing color conversion of the image data input through the input means based on preset lattice point data, color distribution recognition means for recognizing a color distribution in a color space and position information in an image space of the image data input through the input means, grouping means for sorting the color distribution into one or more groups based on the position information recognized by the color distribution recognition means, and means for determining the lattice point data for the one or more multi-dimensional color conversion tables based on the groups into which the color distribution is sorted by the grouping means, the position information, and the color reproduction range of the output device.
In the invention as in aspect 28, in the image processing apparatus as in aspect 26 or 27, the color distribution recognition means divides the color space into areas and counts the number of pixels belonging to each division area and the grouping means determines whether or not adjacent areas can be combined based on the counts of the adjacent areas provided by the color distribution recognition means and groups the areas in response to the determination result.
In the invention as in aspect 29, in the image processing apparatus as in aspect 26 or 27, the color distribution recognition means divides the color space into areas, counts the number of pixels belonging to each division area, and recognizes pixel distribution information and the grouping means determines whether or not adjacent areas can be combined based on the counts of the adjacent areas provided by the color distribution recognition means and the pixel distribution information and groups the areas in response to the determination result.
In the invention as in aspect 30, in the image processing apparatus as in aspect 26 or 27, the grouping means groups the areas except for color distributions of portions where no degradation of color discrimination is assumed to occur.
In the invention as in aspect 31, in the image processing apparatus as in aspect 26 or 27, the lattice point data determination means determines a color compression method and color compression strength for the group based on a distribution form of a portion where a color portion space on the color space contained in the group is outside the color reproduction range.
In the invention as in aspect 32, in the image processing apparatus as in aspect 26 or 27, the lattice point data determination means determines a color compression method and color compression strength for the group based on a distribution form of a portion where a color portion space on the color space contained in the group is outside the color reproduction range and a distribution form of the group.
In the invention as in aspect 33, in the image processing apparatus as in aspect 27, the lattice point data determination means comprises means for determining position mask information corresponding to the groups into which the color distribution is sorted by the grouping means and storage means for creating and storing a file of the lattice point data corresponding to the position mask information determined by the position mask information determination means for determining the lattice point data by reading the lattice point data based on the position mask information from the storage means.
According to the invention as in aspect 34, there is provided an image processing apparatus comprising means for inputting image data, one or more multi-dimensional color conversion tables for executing color conversion of the image data input through the input means based on preset lattice point data, cluster preparation means for extracting areas on an image space made up of similar color pixels based on the image data input through the input means and preparing clusters based on the extracted areas, cluster analysis means for recognizing color distributions of the clusters prepared by the cluster preparation means and preparing a group, and means for determining lattice point data for the one or more multi-dimensional color conversion tables based on the clusters prepared by the cluster preparation means, the group generated by the cluster analysis means, and a color reproduction range of an output device.
According to the invention as in aspect 35, there is provided an image processing method comprising the steps of recognizing a color distribution in a color space and position information in an image space of input image data and determining one or more color reproduction parameters of the image data based on the color distribution, the position information, and a color reproduction range of an output device.
According to the invention as in aspect 36, there is provided an image processing method comprising the steps of recognizing a color distribution of input image data, sorting the recognized color distribution into one or more groups, and determining a color reproduction parameter of the image data based on the groups and a color reproduction range of an output device.
According to the invention as in aspect 37, there is provided an image processing method comprising the steps of recognizing a color distribution in a color space and position information in an image space of input image data, sorting the color distribution into one or more groups based on the recognized position information, and determining one or more color reproduction parameters of the image data based on the groups and a color reproduction range of an output device.
In the invention as in aspect 38, in the image processing method as in aspect 36 or 37, the color distribution recognition step divides the color space into areas and counts the number of pixels belonging to each division area and the grouping step determines whether or not adjacent areas can be combined based on the counts of the adjacent areas provided by the color distribution recognition step and groups the areas in response to the determination result.
In the invention as in aspect 39, in the image processing method as in aspect 36 or 37, the color distribution recognition step divides the color space into areas, counts the number of pixels belonging to each division area, and recognizes pixel distribution information and the grouping step determines whether or not adjacent areas can be combined based on the counts of the adjacent areas provided by the color distribution recognition step and the pixel distribution information and groups the areas in response to the determination result.
In the invention as in aspect 40, in the image processing method as in aspect 36 or 37, the grouping step groups the areas except for color distributions of portions where no degradation of color discrimination is assumed to occur.
In the invention as in aspect 41, in the image processing method as in aspect 36 or 37, the color reproduction parameter determination step includes determining a color compression method and color compression strength for the group based on a distribution form of a portion where a color portion space on the color space contained in the group is outside the color reproduction range.
In the invention as in aspect 42, in the image processing method as in aspect 36 or 37, the color reproduction parameter determination step includes determining a color compression method and color compression strength for the group based on a distribution form of a portion where a color portion space on the color space contained in the group is outside the color reproduction range and a distribution form of the group.
In the invention as in aspect 43, in the image processing method as in aspect 35 or 37, the color reproduction parameter determination step includes preparing one or more conversion parameters of a first equal color space to a second equal color space and one or more pieces of position mask information each indicating a position on the image space to which the conversion parameter is applied.
According to the invention as in aspect 44, there is provided an image processing method comprising the steps of extracting areas on an image space made up of similar color pixels based on input image data and preparing clusters, recognizing color distributions of the clusters and preparing a group, and determining one or more color reproduction parameters of the image data based on the clusters, the group, and a color reproduction range of an output device.
According to the invention as in aspect 45, there is provided an image processing system comprising an image input device for inputting color image data, an image processing apparatus for performing image processing for color image data input through the image input device, and an image output device for outputting the color image data for which the image processing has been performed by the image processing apparatus, characterized in that the image processing apparatus comprises color distribution recognition means for recognizing a color distribution in a color space and position information in an image space of the color image data, means for determining a color reproduction parameter of the color image data based on the color distribution, the position information, and a color reproduction range of the image output device, and conversion means for performing color reproduction processing for the color image data based on the color reproduction parameter determined by the color reproduction parameter determination means and the position information and outputting the resulting conversion color image data to the image output device and that the image output device outputs a color image based on the conversion color image data output from the image processing apparatus.
According to the invention as in aspect 46, there is provided an image processing system comprising an image input device for inputting color image data, an image processing apparatus for performing image processing for color image data input through the image input device, and an image output device for outputting the color image data for which the image processing has been performed by the image processing apparatus, characterized in that the image processing apparatus comprises color distribution recognition means for recognizing a color distribution of the color image data, grouping means for sorting the color distribution recognized by the color distribution recognition means into one or more groups, means for determining a color reproduction parameter of the color image data based on the groups into which the color distribution is sorted by the grouping means and a color reproduction range of the image output device, and conversion means for performing color reproduction processing for the color image data based on the color reproduction parameter determined by the color reproduction parameter determination means and outputting the resulting conversion color image data to the image output device and that the image output device outputs a color image based on the conversion color image data output from the image processing apparatus.
According to the invention as in aspect 47, there is provided an image processing system comprising an image input device for inputting color image data, an image processing apparatus for performing image processing for color image data input through the image input device, and an image output device for outputting the color image data for which the image processing has been performed by the image processing apparatus, characterized in that the image processing apparatus comprises color distribution recognition means for recognizing a color distribution in a color space and position information in an image space of the color image data, grouping means for sorting the color distribution into one or more groups based on the position information recognized by the color distribution recognition means, means for determining a color reproduction parameter of the color image data based on the groups into which the color distribution is sorted by the grouping means and a color reproduction range of the image output device, and conversion means for performing color reproduction processing for the color image data based on the color reproduction parameter determined by the color reproduction parameter determination means and the position information and outputting the resulting conversion color image data to the image output device and that the image output device outputs a color image based on the conversion color image data output from the image processing apparatus.
According to the invention as in aspect 48, there is provided an image processing system comprising an image input device for inputting color image data, an image processing apparatus for performing image processing for color image data input through the image input device, and an image output device for performing image processing for the color image data based on an image processing instruction given by the image processing apparatus and outputting a color image, characterized in that the image processing apparatus comprises color distribution recognition means for recognizing a color distribution in a color space and position information in an image space of the color image data and means for determining a color reproduction parameter of the color image data based on the color space, the position information, and a color reproduction range of the image output device and that the image output device performs color conversion processing for the color image data based on the color reproduction parameter and the position information output from the image processing apparatus and outputs a color image.
According to the invention as in aspect 49, there is provided an image processing system comprising an image input device for inputting color image data, an image processing apparatus for performing image processing for color image data input through the image input device, and an image output device for performing image processing for the color image data based on an image processing instruction given by the image processing apparatus and outputting a color image, characterized in that the image processing apparatus comprises color distribution recognition means for recognizing a color distribution of the color image data, grouping means for sorting the color distribution recognized by the color distribution recognition means into one or more groups, and means for determining a color reproduction parameter of the color image data based on the groups into which the color distribution is sorted by the grouping means and a color reproduction range of the image output device, and that the image output device performs color conversion processing for the color image data based on the color reproduction parameter output from the image processing apparatus and outputs a color image.
According to the invention as in aspect 50, there is provided an image processing system comprising an image input device for inputting color image data, an image processing apparatus for performing image processing for color image data input through the image input device, and an image output device for performing image processing for the color image data based on an image processing instruction given by the image processing apparatus and outputting a color image, characterized in that the image processing apparatus comprises color distribution recognition means for recognizing a color distribution in a color space and position information in an image space of the color image data, grouping means for sorting the color distribution into one or more groups based on the position information recognized by the color distribution recognition means, and means for determining a color reproduction parameter of the color image data based on the groups into which the color distribution is sorted by the grouping means and a color reproduction range of the image output device, and that the image output device performs color conversion processing for the color image data based on the color reproduction parameter and the position information output from the image processing apparatus and outputs a color image.
According to the invention as in aspect 51, there is provided an image processing system comprising an image input device for inputting color image data, an image processing apparatus for performing image processing for color image data input through the image input device, and an image output device for performing image processing for the color image data based on an image processing instruction given by the image processing apparatus and outputting a color image, characterized in that the image processing apparatus comprises cluster preparation means for extracting areas on an image space made up of similar color pixels based on the color image data and preparing clusters based on the extracted areas, cluster analysis means for recognizing color distributions of the clusters prepared by the cluster preparation means and preparing a group, and means for determining a color reproduction parameter of the color image data based on the clusters prepared by the cluster preparation means, the group generated by the cluster analysis means, and a color reproduction range of the image output device, and that the image output device performs color conversion processing for the color image data based on the color reproduction parameter and the position information output from the image processing apparatus and outputs a color image.